Lee-TM: A Non-trivial Benchmark Suite for Transactional Memory

Transactional Memory (TM) is a concurrent programming paradigm that aims to make concurrent programming easier than finegrain locking, whilst providing similar performance and scalability. Several TM systems have been made available for research purposes. However, there is a lack of a wide range of non-trivial benchmarks with which to thoroughly evaluate these TM systems. This paper introduces Lee-TM, a non-trivial and realistic TM benchmark suite based on Lee’s routing algorithm. The benchmark suite provides sequential, lock-based, and transactional implementations to enable direct performance comparison. Lee’s routing algorithm has several of the desirable properties of a non-trivial TM benchmark, such as large amounts of parallelism, complex contention characteristics, and a wide range of transaction durations and lengths. A sample evaluation shows unfavourable transactional performance and scalability compared to lock-based execution, in contrast to much of the published TM evaluations, and highlights the need for non-trivial TM benchmarks.